Shared reactions within a video communication session

ABSTRACT

Methods and systems provide for the ability to display reactions within a video communication session. First, the system displays, for each of a number of participants within a video communication session, a user interface (UI), with the UI including a reactions session with a variety of selectable visual reactions, a number of participant windows corresponding to participants, and videos of at least a subset of the participants which are displayed within the participant windows. The system receives a selection of a reaction via a video client device connected to the video communication session, where the client device is associated with one of the participants. The system then displays the selected reaction within the UI of at least one of the participants within the corresponding participant windows for a predetermined period of time. Aggregate reactions may be displayed when a number of participants select the same reaction.

FIELD OF INVENTION

The present invention relates generally to digital communication, andmore particularly, to systems and methods providing for the sharing ofvisual reactions within a video communication session.

BACKGROUND

Digital communication tools and platforms have been essential inproviding the ability for people and organizations to communicate andcollaborate remotely, e.g., over the internet. In particular, there hasbeen massive adopted use of video communication platforms allowing forremote video sessions between multiple participants. Videocommunications applications for casual friendly conversation (“chat”),webinars, large group meetings, work meetings or gatherings,asynchronous work or personal conversation, and more have exploded inpopularity.

One of the side effects of such virtual, remote meetings via videocommunication sessions is that videos of participants within a groupsession may not as easily translate the reactions of those participantsto the conversation, due to the nature of windows of participants beingsplit across the available space of a screen. Participants may also wantto share reactions to the conversation without having video enabled, ormay want to share reactions for a presenter to see during a presentationwithout needing to speak and interrupt that presenter. Participants mayalso want to emphasize a particular reaction as a fun way to expressthemselves, in the same way that many enjoy using emojis on messagingservices.

Within some video livestreaming services, users watching the livestreammay be able to share reactions during a livestream. For example, duringa livestream, a chat messaging window is displayed alongside thestreaming content. Someone participating in the chat may react tocontent and select a reaction, which is then displayed. A record of thatreaction is made, and the reaction is displayed during all futureplaybacks of the livestream. This works within the one-to-many nature oflivestreaming content, where reactions can be displayed within or nextto chat messages which appear alongside the video content. However,solutions for many-to-many interactions, such as are typical in groupmeetings and video sessions with multiple participants, are lacking.Rather than the chat experience and emotions surrounding it during alivestream, there is no way of sharing reactions during a group videosessions in order to express ephemeral emotions during a liveconversation between participants.

In addition, there is no solution for a large number of peopleexpressing reactions simultaneously during a many-to-many video session.While a presenter is speaking, for example, others may be quiet or havetheir video feeds disabled, and thus there is often a lack of immediatereaction from the audience to the presenter, which can make thepresenter feel that the audience is unreceptive or bored.

Thus, there is a need in the field of digital communication tools andplatforms to create a new and useful system and method for sharingreactions within a video communication session. The source of theproblem, as discovered by the inventors, is a lack of ability forparticipants to communicate visual reactions quickly in live groupconversations.

SUMMARY

The invention overcomes the existing problems by enabling the abilityfor participants within a video communication session to share visualreactions to the conversation. Such reactions may be able to createlittle expressions of joy, intimacy, or connection in order to bestcapture that particular moment's experience. For example, a family maybe able to meet virtually via a remote video communication session on aplatform. During this special moment, reactions depicting a heart may beused by the family members to express love. This heightens the feelingof intimacy between the family members during that session. Duringprofessional meetings, reactions might be used to silently confirm orsupport various points a participant is making without interruptingthem, or may be used to add some much-needed levity and fun to anotherwise serious meeting. For example, when a 30-minute break isannounced during a webinar session, participants may react with a“party” visual reaction to keep things light-hearted, or to segue intopotential networking or light chatting between participants.

In some embodiments, during group meetings with sometimes dozens or evenhundreds of participants in a many-to-many video session, it may bebeneficial to express aggregated reactions overall within the sessionwhich are culled from many individual reactions from participants. Forexample, when a speaker is presenting, it may be useful feedback toreceive an aggregated reaction depicting hundreds of hearts, so that thespeaker knows the audience is supportive and keeping interested, orhundreds of laugh reactions to show that a joke was received well.

One embodiment relates to a communication system configured to perform anumber of operations. First, the system displays, for each of a numberof participants within a video communication session, a user interface(UI), with the UI including a reactions session with a variety ofselectable visual reactions, a number of participant windowscorresponding to participants, and videos of at least a subset of theparticipants which are displayed within the participant windows. Thesystem receives a selection of a reaction from the variety of reactionsvia a video client device connected to the video communication session,where the client device is associated with one of the participants. Thesystem then displays the selected reaction within the UI of at least oneof the participants within the corresponding participant windows for apredetermined period of time.

In some embodiments, the system receives one or more additionalselections of the reaction from client devices associated withadditional participants. The system then aggregates the selection andthe additional selections of the reaction, and determines whether athreshold number of selections has been met or exceeded to display anaggregate reaction. If the threshold has been met or exceeded, then thesystem displays an aggregate reaction, wherein the selected reaction isdisplayed within the UI as the aggregate reaction representing thereactions of multiple participants within the video communicationsession.

Further areas of applicability of the present disclosure will becomeapparent from the detailed description, the claims and the drawings. Thedetailed description and specific examples are intended for illustrationonly and are not intended to limit the scope of the disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention relates generally to digital communication, andmore particularly, to systems and methods providing for containment ofsensitive data within a communication or messaging platform.

The present disclosure will become better understood from the detaileddescription and the drawings, wherein:

FIG. 1A is a diagram illustrating an exemplary environment in which someembodiments may operate.

FIG. 1B is a diagram illustrating an exemplary computer system that mayexecute instructions to perform some of the methods herein.

FIG. 2 is a flow chart illustrating an exemplary method that may beperformed in some embodiments.

FIG. 3 is a flow chart illustrating an exemplary method that may beperformed in some embodiments.

FIG. 4A is a diagram illustrating one example embodiment of a UI withina video communication session, including a selectable reactions UIelement.

FIG. 4B is a diagram illustrating one example embodiment of a variety ofselectable reactions within a video communication session.

FIG. 4C is a diagram illustrating one example embodiment of a reactionfrom a participant being displayed within a video communication session.

FIG. 4D is a diagram illustrating one example embodiment of reactionsfrom multiple participants being displayed concurrently within a videocommunication session.

FIG. 4E is a diagram illustrating one example embodiment of anaggregated reaction from many participants being displayed within avideo communication session.

FIG. 4F is a diagram illustrating one example embodiment of additionalselectable reactions displayed within a video communication session.

FIG. 4G is a diagram illustrating one example embodiment of an expandedsubmenu of selectable reactions displayed within a video communicationsession.

FIG. 5 is a diagram illustrating an exemplary computer that may performprocessing in some embodiments.

DETAILED DESCRIPTION

In this specification, reference is made in detail to specificembodiments of the invention. Some of the embodiments or their aspectsare illustrated in the drawings.

For clarity in explanation, the invention has been described withreference to specific embodiments, however it should be understood thatthe invention is not limited to the described embodiments. On thecontrary, the invention covers alternatives, modifications, andequivalents as may be included within its scope as defined by any patentclaims. The following embodiments of the invention are set forth withoutany loss of generality to, and without imposing limitations on, theclaimed invention. In the following description, specific details areset forth in order to provide a thorough understanding of the presentinvention. The present invention may be practiced without some or all ofthese specific details. In addition, well known features may not havebeen described in detail to avoid unnecessarily obscuring the invention.

In addition, it should be understood that steps of the exemplary methodsset forth in this exemplary patent can be performed in different ordersthan the order presented in this specification. Furthermore, some stepsof the exemplary methods may be performed in parallel rather than beingperformed sequentially. Also, the steps of the exemplary methods may beperformed in a network environment in which some steps are performed bydifferent computers in the networked environment.

Some embodiments are implemented by a computer system. A computer systemmay include a processor, a memory, and a non-transitorycomputer-readable medium. The memory and non-transitory medium may storeinstructions for performing methods and steps described herein.

By way of illustration, a presenter may be hosting a live webinar on aparticular twentieth-century philosopher. The live webinar is hostedwithin a video communication session of a video communication platform,with the presenter presenting materials and speaking within the virtualroom. The participants number in the hundreds. The presenter speaks foran hour while sharing presentation slides, occasionally pausing to takewater breaks. The audience of hundreds is silent, as the host has mutedall participants other than the speaker. Some participants have theirvideo enabled, but many do not. At times, the speaker makes a joke orlighthearted comment, then pauses for effect.

In previous approaches, the presenter does not receive much in the wayof feedback. He is unsure whether his jokes and comments are receivedwell, as he gets no clear, visible reaction from the audience. They maybe listening with rapt attention, bored, away from the computerentirely, or laughing, but it is difficult for the presenter to gaugethe mood of the room or how his words are being received from moment tomoment. As a result, he feels that the remote video session had a “cold”feeling and feels that he lacked real connection with his audience.

In contrast, with the present approach, participants are able to selectreactions from within the individual UIs on their devices. The reactionsare displayed in their participant windows to other participants. Frommoment to moment, non-presenting participants can see the reactions ofother participants. The presenter has his UI configured such that hedoes not see individual reactions from participants, which may bevisually distracting while he is presenting; instead, he sees aggregatereactions when a number of people in the audience select the samereaction concurrently within a given window of time. Thus, even thoughnon-presenters are muted and many have their video disabled, thepresenter occasionally sees an aggregate reaction displayed on his UIwhen the overall sentiment of the audience sways towards a particularreaction.

For example, upon the presenter sharing a particular anecdote about thephilosopher that the audience finds touching, the presenter sees anaggregate reaction of hearts appear within the display during or shortlyafter the anecdote. When the presenter makes a joke and pauses foreffect, he sees an aggregate reaction of laughing faces on his screen,which gives him valuable information that the audience is payingattention on the whole and that his jokes are being received well. Inthe moment, this encourages the presenter that he has the audienceengaged, and causes him to conclude the presentation with a big finish.Through constant visual notification of the audience's reactions in theaggregate, he comes away from the presentation feeling that he connectedwith his audience and that the presentation went well. Thenon-presenting audience, in turn, feels that they shared particularsentiments with others in the crowd, and feel more connected havingshared their own sentiments. They are able to express laughter orapplause without causing disruption of the presentation and withoutneeding to be seen on video. They also received information aboutindividual participants and what reactions they had during variousmoments of the presentation.

I. Exemplary Environments

FIG. 1A is a diagram illustrating an exemplary environment in which someembodiments may operate. In the exemplary environment 100, a firstuser's client device 150 and one or more additional users' clientdevice(s) 160 are connected to a processing engine 102 and, optionally,a communication platform 140. The processing engine 102 is connected tothe communication platform 140, and optionally connected to one or morerepositories and/or databases, including a participants repository 130,reactions repository 132, and/or a settings repository 134. One or moreof the databases may be combined or split into multiple databases. Thefirst user's client device 150 and additional users' client device(s)160 in this environment may be computers, and the communication platformserver 140 and processing engine 102 may be applications or softwarehosted on a computer or multiple computers which are communicativelycoupled via remote server or locally.

The exemplary environment 100 is illustrated with only one additionaluser's client device, one processing engine, and one communicationplatform, though in practice there may be more or fewer sending clientdevices, receiving client devices, processing engines, and/orcommunication platforms. In some embodiments, the sending client device,receiving client device, processing engine, and/or communicationplatform may be part of the same computer or device.

In an embodiment, the processing engine 102 may perform the exemplarymethod of FIG. 2A, the exemplary method of FIG. 3 , or other methodherein and, as a result, provide the ability for users to sharereactions within a video communication session. In some embodiments,this may be accomplished via communication with the first user's clientdevice, additional users' client device(s), processing engine,communication platform, and/or other device(s) over a network betweenthe device(s) and an application server or some other network server. Insome embodiments, the processing engine 102 is an application, browserextension, or other piece of software hosted on a computer or similardevice, or is itself a computer or similar device configured to host anapplication, browser extension, or other piece of software to performsome of the methods and embodiments herein.

The first user's client device 150 and additional users' clientdevice(s) 160 are devices with a display configured to presentinformation to a user of the device. In some embodiments, the clientdevices present information in the form of a user interface (UI) withmultiple selectable UI elements or components. In some embodiments, theclient devices 150 and 160 are configured to send and receive signalsand/or information to the processing engine 102 and/or communicationplatform 140. In some embodiments, the client devices are computingdevices capable of hosting and executing one or more applications orother programs capable of sending and/or receiving information. In someembodiments, the client device may be a computer desktop or laptop,mobile phone, virtual assistant, virtual reality or augmented realitydevice, wearable, or any other suitable device capable of sending andreceiving information. In some embodiments, the processing engine 102and/or communication platform 140 may be hosted in whole or in part asan application or web service executed on the client devices 150 and/or160. In some embodiments, one or more of the communication platform 140,processing engine 102, and client devices 150 and 160 may be the samedevice. In some embodiments, the first user's client device 150 isassociated with a first user account within a video platform, and theadditional users' client device(s) 160 are associated with additionaluser account(s) within a video platform.

In some embodiments, optional repositories can include one or more of aparticipants repository 130, reactions repository 132, and/or settingsrepository 134. The optional repositories function to store and/ormaintain, respectively, participant information associated with a videocommunication session on the communication platform 140, selectablereactions within the video communication session, and settings of thevideo communication session. The optional database(s) may also storeand/or maintain any other suitable information for the processing engine102 or communication platform 140 to perform elements of the methods andsystems herein. In some embodiments, the optional database(s) can bequeried by one or more components of system 100 (e.g., by the processingengine 102), and specific stored data in the database(s) can beretrieved.

Communication platform 140 is a platform configured to facilitate videocommunication between two or more parties, such as within aconversation, video conference or meeting, message board or forum,virtual meeting, or other form of digital communication. The videocommunication session may be one-to-many (e.g., a speaker presenting tomultiple attendees), one-to-one (e.g., two friends speaking with oneanother), or many-to-many (e.g., multiple participants speaking witheach other in a group video setting).

FIG. 1B is a diagram illustrating an exemplary computer system 150 withsoftware modules that may execute some of the functionality describedherein.

User interface display module 152 functions to display a UI for each ofthe participants within the video communication session, including atleast a reactions UI element with selectable reactions, participantwindows corresponding to participants, and videos displayed withinparticipant windows.

Video display module 154 functions to display the videos for at least asubset of the participants, which may appear as live video feeds foreach participant with video enabled.

Reaction selection module 156 functions to receive, from a clientdevice, a selection of a reaction from the available reactions withinthe reactions UI element.

Reaction display module 158 functions to display the selected reactionwithin the UI of at least one of the plurality of participants withinthe corresponding participant window for a predetermined period of time.

Optional threshold determination module 160 functions to determine thata threshold number of selections of a particular reaction fromparticipants has been met or exceeded.

Aggregate reaction display module 162 functions to aggregate the initialselection and the additional selections of the reaction, and if thethreshold has been met or exceeded, functions to display the aggregatereaction within the UI of at least one of the plurality of participantsfor a predetermined period of time.

The above modules and their functions will be described in furtherdetail in relation to an exemplary method below.

II. Exemplary Method

FIG. 2 is a flow chart illustrating an exemplary method that may beperformed in some embodiments.

At step 202, the system displays a user interface for each of aplurality of participants within a video communication session. The UIincludes at least: a reactions section with a number of selectablevisual reactions; participant windows corresponding to participants; andvideo for each of at least a subset of the participants to be displayedwithin the corresponding participant window for the participant.

The UI to be displayed relates to the communication platform 140, andmay represent a “video window”, such as a window within a GUI thatdisplays a video between a first participant, with a user account withinthe video platform, and one or more other user accounts within the videoplatform. The first participant is connected to the video communicationsession via a client device. The UI includes a number of selectable UIelements. For example, one UI may present selectable UI elements alongthe bottom of a communication session window, with the UI elementsrepresenting options the participant can enable or disable within thevideo session, settings to configure, and more. For example, UI elementsmay be present for, e.g., muting or unmuting audio, stopping or startingvideo of the participant, sharing the participant's screen with otherparticipants, recording the video session, and/or ending the videosession. One example of a communication interface within a communicationplatform is illustrated in FIG. 4A, which will be described in furtherdetail below.

One included UI element is a selectable reactions UI element. An exampleof this secondary UI element is illustrated in FIG. 4B, which will bedescribed in further detail below. Examples of selectable reactions mayinclude, e.g., clapping hands, a raised hand, a thumbs up symbolindicating approval, a thumbs down symbol indicating disapproval, aheart symbol, a laughing face, a surprised face, a party symbol (e.g., aparty horn and confetti), a smiling face, a frowning face, a cryingface, or any other suitable visual reaction. In some embodiments, asubset or the full set of reactions may be emojis. In some embodiments,emojis may be standard Unicode emoji characters and/or sequences, orother universal or standardized emojis. In some embodiments, emojis maybe non-standard emojis. In some embodiments, reactions may be animated.For example, they may be in animated GIF format or any other formatwhich allows animated images to appear on the screen. In someembodiments, reactions may be static images. In some embodiments,reactions may be audio segments or clips rather than visual reactions.In some embodiments, reactions may be drawings generated by participantsvia a touch, pen, or stylus input.

In some embodiments, one or more of the selectable reactions may beprovided by a participant within the video session or a third party. Insome embodiments, selectable reactions are extensible, and may besubmitted or uploading by users of the video platform. For example, theplatform may be configured to allow different emoji providers to addtheir own emoji-based “sticker packs”, which can be chosen by aparticipant and then incorporated into the set of available visualreactions for that participant during the video communication session.In some embodiments, users may be able to upload their own reactions. Insome embodiments, users may be able to purchase premium reactions ordownload free reactions from a separate online store interface.

In some embodiments, there may be a different set or multiple sets ofreactions depending on the video session in question, whether the videosession may be categorized or tagged as, e.g., a formal business meetingor an information get-together between friends, etc. In one informalvideo session, a particularly fun or silly set of reactions may beavailable, while in a formal session with potential business investors,the reactions may be less silly and more appropriate for a businesssetting. In some embodiments, participants, administrators, hosts,and/or owners of corporate accounts can have the ability to configurewhich set or sets of reactions are to be selectable for a given videosession.

Another portion of the UI displays a number of participant windows. Theparticipant windows correspond to the multiple participants in the videocommunication session. Each participant is connected to the videocommunication session via a client device. In some embodiments, theparticipant window may include video, such as, e.g., video of theparticipant or some representation of the participant, a room theparticipant is in or virtual background, and/or some other visuals theparticipant may wish to share (e.g., a document, image, animation, orother visuals). In some embodiments, the participant's name (e.g., realname or chosen username) may appear in the participant window as well.One or more participant windows may be hidden within the UI, andselectable to be displayed at the user's discretion. Variousconfigurations of the participant windows may be selectable by the user(e.g., a square grid of participant windows, a line of participantwindows, or a single participant window). The participant windows arealso configured to display reactions from the participant in question,as will be discussed in further detail below. Some participant windowsmay not contain any video, for example, if a participant has disabledvideo or does not have a connected video camera device (e.g. a built-incamera within a computer or smartphone, or an external camera deviceconnected to a computer).

The videos displayed for at least a subset of the participants appearwithin each participant's corresponding participant window. Video maybe, e.g., a live feed which is streamed from the participant's clientdevice to the video communication session. In some embodiments, thesystem receives video content depicting imagery of the participant, withthe video content having multiple video frames. The system providesfunctionality for a participant to capture and display video imagery toother participants. For example, the system may receive a video streamfrom a built-in camera of a laptop computer, with the video streamdepicting imagery of the participant.

At step 220, the system receives, from a client device, a selection of areaction from the number of selectable reactions within the reactions UIelement. The client device in question may be, e.g., the first user'sclient device 150, where the first user is a participant of the videosession who selected the reaction from the reactions UI element. In someembodiments, the reactions UI element may be selected by a participantby, e.g., clicking or holding down a mouse button or other component ofan input device, tapping or holding down on the UI element with afinger, stylus, or pen, hovering over the UI element with a mouse orother input device, or any other suitable form of selecting a UIelement. In some embodiments, upon selecting the UI element, a subwindow or other secondary UI element appears which displays a variety ofselectable visual reactions. Upon selecting the desired visual reaction,the selection is sent to the system (e.g., the processing engine 102) tobe processed.

In some embodiments, the selection may be performed at the client deviceby one or more participant gestures or expressions. For example, thesystem may be configured such that the user can use sign languagegestures to select reactions, or basic gestures such as a thumbs up. Insome embodiments, the camera captures the gesture, analyzes it andclassifies as the gesture in question, and then selects the appropriatereaction in response. In some embodiments, expressions may be usedinstead of gestures. For example, when a participant smiles, the cameracaptures the participant smiling, analyzes the image, classifies it as asmile, then selects the appropriate smiling face reaction in response.In some embodiments, the participant's gesture may be clapping his orher hands. The clapping may be detected by the camera, by the audiocapture device, or both. For example, when one person claps, the systemmay interpret the audio as clapping and automatically select a clappingreaction for the participant who is clapping. In some embodiments,multiple people clapping may trigger an aggregate reaction. Aggregatereactions will be described in further detail below with respect to FIG.3 and the accompanying description.

At step 230, the system displays the selected reaction within the UI ofat least one of the participants within the corresponding participantwindow for a predetermined period of time. In some embodiments, theselected reaction is displayed within the participant windowcorresponding to the participant who selected the reaction. For example,if Participant 1 selects a clapping hands reaction, then a clappinghands reaction will appear in the participant window for Participant 1.In some embodiments, the reaction is visible in a portion of theparticipant window, e.g., the lower right corner. In some embodiments,the reaction may be static, while in others, the reaction may beanimated. One example of a selected reaction being displayed within a UIis illustrated in FIG. 4C, which will be described in further detailbelow.

In some embodiments, the reaction is visible to all participants whoseUI is currently showing the participant window of the participant whoselected the UI reaction. In some embodiments, the reaction is visibleto all participants regardless of whether the appropriate participantwindow is visible on the UI. For example, some participants may see aclapping hands reaction appear from one corner of the UI, or in somespecific section of the UI. In some embodiments, the reaction may havethe participant's name or username attached to it. The name or usernamemay be visible immediately, visible upon clicking on the reaction, orvisible upon an input device hovering over the reaction.

In some embodiments, an administrator, host, or settings coordinator forthe video communication session may have disabled reactions from beingdisplayed for some or all participants. In some embodiments, this maylead to the reaction not being displayed despite a participantsselecting a reaction. In other embodiments, the participants will not beable to select the reactions UI element as it is greyed out or otherwisedisabled. In some embodiments, participants themselves may have theoption to disable reactions within their own UI. This may be useful forparticipants who find the reactions to be distracting, for example.

The amount of time in which the reaction is displayed within a givenparticipant window is predetermined. In some embodiments, the time ispredetermined based on a default time setting. For example, allreactions may appear displayed for five seconds by default beforedisappearing. In some embodiments, this default time window may bealtered by an administrator, host, or owner of a corporate account in aseparate settings or preferences user interface. In some embodiments,the amount of time a share reaction is displayed (i.e., reaction periodtime) may be dynamically determined based on one or more factors orcriteria. Factors may include, for example, whether the participant whosubmitted the reaction is muted or speaking; whether other participantsare muted or speaking; whether the participant who submitted thereaction has video enabled or disabled; whether there's a pause ormoment of silence in the audio; whether the reaction was selected by amomentary click or tap, or an input indicating a longer interaction,such as holding a mouse button, a long press of a finger on a touchscreen, or similar; whether an individual participant has indicated apreference for longer or shorter times for reactions to be displayed;the number of participants; the number of concurrently displayedreactions; a determined engagement level of the participants; or anyother suitable factor for dynamically determining the time a reaction isdisplayed within the participant window.

In some embodiments, a selected reaction may be displayed within a givenparticipant window while one or more other reactions are displayed inother participant windows. Participants may select reactions to bedisplayed in real time or substantially real time independently of otherparticipants selecting reactions. One example of such simultaneous orconcurrent display of multiple reaction is illustrated in FIG. 4D, whichwill be described in further detail below.

In some embodiments, while a participant is presenting materials orspeaking at length in a video session, the presenter may be speciallyalerted or notified if one or more participants selects a “raised hand”or other reaction indicating that the user has a question or isrequesting to comment. This can be benefit teachers, webinar hosts, andothers who want to be notified when participants have questions orcomments or otherwise want to participate in some active way during apresentation. In some embodiments, the presenter may have the option toturn this alert or notification on or off within the presenter'ssettings or preference for the video session. The alert or notificationmay be visual, audio-based, or both, or may be, e.g., a pushnotification on a separate mobile device associated with the presenter.

FIG. 3 is a flow chart illustrating an optional exemplary method thatmay be performed in some embodiments. In some embodiments, the optionalexemplary method begins at the point where FIG. 2 , step 230 ends, i.e.,after the system displays the selected reaction within one or moreparticipant UIs. In other embodiments, the optional exemplary methodbegins at the end of FIG. 2 , step 220, and step 230 is skipped. In yetother embodiments, at least part of the exemplary method is performedconcurrently to one or more steps of FIG. 2 .

At optional step 310, the system receives one or more additionalselections of the reaction from client devices associated withadditional participants. For example, in some embodiments, the systemreceives the additional selections of reactions from participants whilethe system also receives the initial selection from the participant inFIG. 2 . Each participants may send an additional selection of thereaction from their own UI by navigating to the reactions UI element andselecting the reaction in question. Thus, for example, while the firstparticipant selects a heart emoji, other participants may besimultaneously selecting heart emojis as well via the reactions UIelement in their own UIs.

At optional step 320, the system aggregates the selection and theadditional selections of the reaction. In some embodiments, thisaggregation may be performed by adding the initial selection of thereaction with the number of additional selections of the reaction. Forexample, if there are 7 additional heart reactions, then the systemaggregates the reactions by adding 7+1 for the first selected reactionto obtain 8 heart reactions. In some embodiments, the aggregationcontinues for a predetermined period of time, then resets after thattime has passed. For example, the system may aggregate heart reactionswhich have been selected over a 10 second period of time, then reset to0 after the 10 seconds has passed. In some embodiments, there is apredetermined time window for which aggregations are totaled. Forexample, for any given 10 second window, reactions are aggregated withinthat window. The window is constantly shifting, and any reactions whichwere selected outside of the 10 second window are de-aggregated.

At optional step 330, the system determines that a threshold number ofselections has been met or exceeded to display an aggregate reaction.The number of selections is indicated by the aggregation of reactions instep 320. In some embodiments, the threshold is predetermined as a setnumber. For example, one embodiment may have a threshold set to 10 for avideo communication session. Upon 10 participants concurrently selectingthe same reaction (e.g., a heart reaction) in the video communicationsession, the threshold is exceeded. In some embodiments, the thresholdmay be set by an administrator, host, or owner of a corporate account.In some embodiments, the threshold may be dynamically determined basedon one or more factor. Factors may include, e.g., the number ofparticipants in the session; the number of participants in the sessionwho are actively participating (e.g., speaking regularly orsemi-regularly throughout the session); or any other suitable factor.

In some embodiments, the threshold determines whether the videocommunication session is classified as a large scale group meeting (or“large group meeting”). Large group meetings—such as one-to-many videosessions or many-to-many video sessions with several participants—can beconfigured to have specific settings for reactions, among other aspectsof the session experience. In some embodiments, large group meetings maybe configured to have specific predetermined thresholds for aggregatereactions. In some embodiments, they may also have aggregate reactionsenabled by default, whereas non-large group meetings may have aggregatereactions disabled by default.

In some embodiments, when the video session is determined to be a largegroup meeting, the video communication session's settings are configuredor reconfigured such that aggregate reactions are to be displayedinstead of non-aggregate reactions from individual participants. Thatis, rather than displaying all individual reactions from groupparticipants, which may present clutter, confusion, and too muchon-screen movement and change in a session with many participants, onlyaggregate reactions are displayed to give an overall feeling orsentiment of the group as a whole.

In some embodiments, the threshold is determined to have been met orexceeded when a determined engagement level of the participants crossesa certain threshold. Engagement level may be determined dynamicallybased on, for example, the amount of active speakers or presenterswithin the session, the amount of participants with video enabled, theamount of participants who are selecting reactions during the session,audio analysis which determines the participants are laughing as a groupor clapping as a group, or any other suitable factor.

At optional decision point 340, if the threshold has not been reached(i.e., met or exceeded), then the system continues to determine at step330 whether the threshold has been reached while participants continuewith the video session. If the threshold has been reached, then thesystem proceeds to optional step 350.

At optional step 350, the system displays an aggregate reaction. Theselected reaction displays within the UI of one or more participants asan aggregate reaction. The aggregate reaction represents the reactionsof multiple participants in the video. In some embodiments, theaggregate reaction appears as several iterations of the selectedreaction displayed within the UI. These iterations may appear inparticipant windows, outside of participant windows, or both inside andoutside of participant windows. In some embodiments, the system displaysan animated version of the shared reaction. In some embodiments, thesystem displays a larger version of the shared reaction on the screen.Many other possibilities for displaying aggregate reactions may becontemplated. In some embodiments, the aggregate reaction is displayedon the screen for a longer time than individual reactions are displayed.One example of an aggregate reaction being displayed within a UI isillustrated in FIG. 4E, which will be described in further detail below.

In some embodiments, if there are one or more presenters within a videocommunication session, i.e., participants who are primary speakers, whomay be presenting materials and/or sharing their screen, or otherwiseacting as the primary participant of the session, then the system mayconfigure the UI of that presenter to only display aggregate reactionsrather than individual reactions, while the UIs of other participantsare configured to display non-aggregate, individual reactions. In thisway, the presenter may see only the overall sentiment or feeling of theroom at a glance without being distracted, while other participants seeindividual reactions.

FIGS. 4A-4G are diagrams illustrating various aspects of the systems andmethods herein through different example embodiments.

FIG. 4A is a diagram illustrating one example embodiment of a UI withina video communication session, including a selectable reactions UIelement.

User interface 400 depicts a UI that a particular participant is viewingon a screen of the participant's client device. For purposes of theexample embodiments of FIGS. 4A-4E, the UI belongs to the participant inthe top left corner. Four participant windows are displayed within theUI, arranged in a 4×4 grid. Within each participant window is a video.The video in each of the participant windows is a live video feedcaptured via a camera or other device that is either built into orconnected to the client device of that participant, then streamed to theUIs of participants. Also appearing in the bottom left corner of eachparticipant window is a name of the participant, as well as an iconindicating that the participant has their audio muted, if applicable. Inthe top right, a selectable UI element allows a participant to togglebetween a full screen view and non-full-screen view. To the right, achat or messaging section of the UI provides participants to entermessages to be displayed while the video communication session proceeds.

A bar at the bottom of the UI present a number of selectable UI elementswithin the UI. These elements include Mute, Stop Video, Security,Participants, Chat, Share Screen, Polling, Record, Closed Caption,Reactions, More, and End. The Reactions UI element 420 is selectable bya participants through an input device, i.e., a user clicking on the UIelement while a mouse pointer is pointed at the UI element.

FIG. 4B is a diagram illustrating one example embodiment of a variety ofselectable reactions within a video communication session.

FIG. 4B is identical to FIG. 4A, except the participant has selected theReactions UI element 420. A sub element or submenu 430 appears after theparticipant selects the UI element. The sub element 430 appears in thespace directly above the Reactions UI element 420. The sub element 430displays a variety of selectable reactions, including a clapping handreaction, a thumbs up reaction, a heart reaction, a laughing facereaction, a surprised face reaction, and a party reaction. A participantmay navigate the input device to one of the selectable reactions andselect it with the input device.

FIG. 4C is a diagram illustrating one example embodiment of a reactionfrom a participant being displayed within a video communication session.

FIG. 4C is identical to FIG. 4B, except the participant has selected aheart reaction from the sub element 430. The sub element 430 disappears,and the selected reaction is displayed in the lower right corner of theparticipant's UI. In this case, since the user interface 400 belongs tothe participant in the top left participant window, when the participantselected the heart reaction, it appeared in the participant's ownvisible participant window. In some embodiments, if the participant hasselected to not view video of himself or herself, then the selectedreaction may still be briefly shown in some portion of the UI toindicate that the reaction was successfully displayed to otherparticipants.

FIG. 4D is a diagram illustrating one example embodiment of reactionsfrom multiple participants being displayed concurrently within a videocommunication session.

FIG. 4D is identical to FIG. 4C, except multiple participants haveselected reactions to be displayed, not just the participant in the topleft. Thus, two participant windows display heart reactions, while oneparticipant window displays a party reaction. In some embodiments,multiple reactions can be displayed concurrently and independently ofeach other. In some embodiments, such individual reactions may beconcurrently and independently displayed, even in a session with manyparticipants. For example, even a session filled with 500 or moreparticipants may be configured to display individual reactions ofparticipants.

FIG. 4E is a diagram illustrating one example embodiment of anaggregated reaction from many participants being displayed within avideo communication session.

FIG. 4E is identical to FIG. 4C, except multiple participants haveselected the same reaction. The reaction has been aggregated to a totalof 4, which crosses a predetermined threshold for displaying anaggregate reaction. For example, the default threshold for the sessionmay be three-fourths of participants. At the point three out of fourparticipants select the heart reaction within a predetermined window oftime, e.g. 10 seconds, the threshold is reached and the aggregatereaction is displayed.

Within the example embodiment, the aggregate reaction is displayed as anumber of heart reaction images which move or float across the screen.This indicates that enough participants in the session have reacted inthe same way that the overall sentiment or mood of the room can becaptured by the heart reaction. The aggregate reaction does not displayin any one participant window, but rather is displayed both inside andoutside of participant windows, across the UI. Many other styles ofaggregate reactions can be displayed based on preferences and needs. Insome embodiments, an administrator, host, or owner of a corporateaccount can configure the settings and preference for the videocommunication session to select a particular style of aggregatereactions, or to turn off aggregate reactions entirely.

FIG. 4F is a diagram illustrating one example embodiment of additionalselectable reactions displayed within a video communication session.

FIG. 4F is identical to FIG. 4B, except that the variety of selectablereactions 430 displayed within FIG. 4B has been expanded to a largervariety of selectable reactions 440. In addition to the six selectablereactions illustrated in FIG. 4B, there are a number of additionalreactions and UI elements which can be selected. There is also aselectable ellipses (“ . . . ”) UI element. Upon clicking on theellipses UI element, a further expanded submenu of selectable reactionscan be displayed, as will be illustrated in FIG. 4G.

FIG. 4G is a diagram illustrating one example embodiment of an expandedsubmenu of selectable reactions displayed within a video communicationsession.

Upon clicking on the ellipses UI element in FIG. 4F, an expanded submenuof selectable reactions 450 is displayed. In some embodiments, thesubmenu includes a search field UI element which allows a user to entercharacters or words to search for the selectable reaction they wouldlike to use. In the example illustrated, a full set of emojis isselectable within the expanded submenu, separated into categories suchas “Smileys & People”). In some embodiments, the user may also selectone of a variety of skin tones for some of the selectable reactions,such as reactions that depict faces or hands.

FIG. 5 is a diagram illustrating an exemplary computer that may performprocessing in some embodiments. Exemplary computer 500 may performoperations consistent with some embodiments. The architecture ofcomputer 500 is exemplary. Computers can be implemented in a variety ofother ways. A wide variety of computers can be used in accordance withthe embodiments herein.

Processor 501 may perform computing functions such as running computerprograms. The volatile memory 502 may provide temporary storage of datafor the processor 501. RAM is one kind of volatile memory. Volatilememory typically requires power to maintain its stored information.Storage 503 provides computer storage for data, instructions, and/orarbitrary information. Non-volatile memory, which can preserve data evenwhen not powered and including disks and flash memory, is an example ofstorage. Storage 503 may be organized as a file system, database, or inother ways. Data, instructions, and information may be loaded fromstorage 503 into volatile memory 502 for processing by the processor501.

The computer 500 may include peripherals 505. Peripherals 505 mayinclude input peripherals such as a keyboard, mouse, trackball, videocamera, microphone, and other input devices. Peripherals 505 may alsoinclude output devices such as a display. Peripherals 505 may includeremovable media devices such as CD-R and DVD-R recorders/players.Communications device 506 may connect the computer 100 to an externalmedium. For example, communications device 506 may take the form of anetwork adapter that provides communications to a network. A computer500 may also include a variety of other devices 504. The variouscomponents of the computer 500 may be connected by a connection mediumsuch as a bus, crossbar, or network.

Some portions of the preceding detailed descriptions have been presentedin terms of algorithms and symbolic representations of operations ondata bits within a computer memory. These algorithmic descriptions andrepresentations are the ways used by those skilled in the dataprocessing arts to most effectively convey the substance of their workto others skilled in the art. An algorithm is here, and generally,conceived to be a self-consistent sequence of operations leading to adesired result. The operations are those requiring physicalmanipulations of physical quantities. Usually, though not necessarily,these quantities take the form of electrical or magnetic signals capableof being stored, combined, compared, and otherwise manipulated. It hasproven convenient at times, principally for reasons of common usage, torefer to these signals as bits, values, elements, symbols, characters,terms, numbers, or the like.

It should be borne in mind, however, that all of these and similar termsare to be associated with the appropriate physical quantities and aremerely convenient labels applied to these quantities. Unlessspecifically stated otherwise as apparent from the above discussion, itis appreciated that throughout the description, discussions utilizingterms such as “identifying” or “determining” or “executing” or“performing” or “collecting” or “creating” or “sending” or the like,refer to the action and processes of a computer system, or similarelectronic computing device, that manipulates and transforms datarepresented as physical (electronic) quantities within the computersystem's registers and memories into other data similarly represented asphysical quantities within the computer system memories or registers orother such information storage devices.

The present disclosure also relates to an apparatus for performing theoperations herein. This apparatus may be specially constructed for theintended purposes, or it may comprise a general purpose computerselectively activated or reconfigured by a computer program stored inthe computer. Such a computer program may be stored in a computerreadable storage medium, such as, but not limited to, any type of diskincluding floppy disks, optical disks, CD-ROMs, and magnetic-opticaldisks, read-only memories (ROMs), random access memories (RAMs), EPROMs,EEPROMs, magnetic or optical cards, or any type of media suitable forstoring electronic instructions, each coupled to a computer system bus.

Various general purpose systems may be used with programs in accordancewith the teachings herein, or it may prove convenient to construct amore specialized apparatus to perform the method. The structure for avariety of these systems will appear as set forth in the descriptionabove. In addition, the present disclosure is not described withreference to any particular programming language. It will be appreciatedthat a variety of programming languages may be used to implement theteachings of the disclosure as described herein.

The present disclosure may be provided as a computer program product, orsoftware, that may include a machine-readable medium having storedthereon instructions, which may be used to program a computer system (orother electronic devices) to perform a process according to the presentdisclosure. A machine-readable medium includes any mechanism for storinginformation in a form readable by a machine (e.g., a computer). Forexample, a machine-readable (e.g., computer-readable) medium includes amachine (e.g., a computer) readable storage medium such as a read onlymemory (“ROM”), random access memory (“RAM”), magnetic disk storagemedia, optical storage media, flash memory devices, etc.

In the foregoing disclosure, implementations of the disclosure have beendescribed with reference to specific example implementations thereof. Itwill be evident that various modifications may be made thereto withoutdeparting from the broader spirit and scope of implementations of thedisclosure as set forth in the following claims. The disclosure anddrawings are, accordingly, to be regarded in an illustrative senserather than a restrictive sense.

What is claimed is:
 1. A communication system comprising one or moreprocessors configured to perform the operations of: displaying, for eachof a plurality of participants within a video communication session, auser interface (UI), the UI comprising: a reactions section comprising aplurality of selectable visual reactions, a plurality of participantwindows corresponding to the plurality of participants in the videocommunication session, wherein each participant is connected to thevideo communication session via a client device, and a video for each ofat least a subset of the participants, wherein the video is displayedwithin the corresponding participant window for the participant;receiving a selection of a reaction from the plurality of reactions viaa client device connected to the video communication session, whereinthe client device is associated with a participant from the plurality ofparticipants; receiving one or more additional selections of thereaction from client devices associated with additional participantsfrom the plurality of participants: determining an aggregate reactionbased on the selection of the reaction and the one or more additionalselections, the determining comprising: for a predetermined period oftime, aggregating reactions received from the plurality of clientdevices; and selecting a group display reaction where the number ofaggregated reactions meets or exceeds a predetermined threshold value,the predetermined threshold value being dynamically determined based ona number of the plurality of participants that are active participants;and providing for display the group display reaction for the UI of atleast one of the plurality of participants within the correspondingparticipant window for a predetermined period of time, the group displayreaction comprising multiple graphical icons of the same type.
 2. Thesystem of claim 1, further comprising: determining that the plurality ofparticipants within the video communication session meets or exceeds athreshold number of participants to be classified as a large scale groupmeeting; and classifying the video communication session as a largescale group meeting such that the aggregate reaction is to be displayedinstead of non-aggregate reactions from individual participants.
 3. Thesystem of claim 1, wherein the received selection of the reaction isdisplayed within the UI of one or more presenting participants withinthe video communication session as an aggregate reaction, and whereinthe received selection of the reaction is displayed within the UI of oneor more non-presenting participants within the video communicationsession as a plurality of non-aggregate reactions.
 4. The system ofclaim 1, wherein the selected reaction is displayed in an animatedfashion.
 5. The system of claim 1, wherein the selectable reactionscomprise one or more of: an animated GIF, a static image, an audiosegment, and/or a drawing generated via a touch, pen, or stylus input.6. The system of claim 1, wherein the selected reaction visuallyindicates that the participant is requesting to comment or ask aquestion.
 7. The system of claim 1, wherein the predetermined period oftime has been configured by an administrator or host of the videocommunication session.
 8. The system of claim 1, wherein anadministrator or host of the video communication session has enabled,via a settings interface, the ability for reactions to be displayedwithin the UIs of participants.
 9. The system of claim 1, wherein eachof one or more of the participants within the video communicationsession has enabled, via a settings interface, the ability for reactionsto be displayed within the UI for that participant.
 10. The system ofclaim 1, wherein the selection of the reaction from the client device isgenerated from one or more participant gestures captured at the clientdevice.
 11. The system of claim 1, wherein a plurality of reactions aredisplayed simultaneously within a plurality of participant windows. 12.The system of claim 1, wherein the plurality of selectable reactions inthe reactions section is an extensible set of reactions comprising oneor more reactions sourced from one or more reaction providers.
 13. Thesystem of claim 1, wherein the reaction criteria comprise one or moreof: number of participants, number of concurrently displayed reactions,and/or a determined engagement level of the participants.
 14. A methodfor displaying reactions within a video communication session,comprising: displaying, for a plurality of participants within a videocommunication session, a video and a user interface (UI), the UIcomprising: a reactions section comprising a plurality of selectablevisual reactions to the video, and a plurality of participant windowscorresponding to the plurality of participants in the videocommunication session, wherein each participant is connected to thevideo communication session via a client device; receiving a selectionof a reaction from the plurality of reactions via a client deviceconnected to the video communication session, wherein the client deviceis associated with a participant from the plurality of participants;receiving one or more additional selections of the reaction from clientdevices associated with additional participants from the plurality ofparticipants: determining an aggregate reaction based on the selectionof the reaction and the one or more additional selections, thedetermining comprising: for a predetermined period of time, aggregatingreactions received from the plurality of client devices; and selecting agroup display reaction where the number of aggregated reactions meets orexceeds a predetermined threshold value, the predetermined thresholdvalue being dynamically determined based on a number of the plurality ofparticipants that are active participants; and providing for display thegroup display reaction for the UI of at least one of the plurality ofparticipants for a predetermined period of time, the group displayreaction comprising multiple graphical icons of the same type.
 15. Themethod of claim 14, wherein the selected reaction is displayed within aparticipant window from the plurality of participant windows, whereinthe participant window corresponds to the participant.
 16. The method ofclaim 14, wherein the plurality of selectable reactions in the reactionssection is an extensible set of reactions comprising one or morereactions sourced from one or more reaction providers.
 17. The method ofclaim 14, wherein the predefined period of time is dynamicallydetermined based on one or more reaction period criteria.
 18. The methodof claim 14, wherein the UI of the at least one of the plurality ofparticipants is configured to display the group display reaction and notan individual reaction based on the selection of the reaction or the oneor more additional selections.
 19. The method of claim 14, wherein theUI of the at least one of the plurality of participants is configured todisplay the group display reaction for a longer time than a reaction forthe selection of the reaction or the one or more additional selections.20. The method of claim 14, wherein the UI of the at least one of theplurality of participants is configured to display the group displayreaction both inside and outside of the plurality of participantwindows, and display an individual reaction based on the selection ofthe reaction or the one or more additional selections within aparticipant window associated with a participant initiating theindividual reaction.